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SUMMARY 

The  purpose  of  this  report  is  to  discuss  a concept,  termed  the  Data  Network 
Referee  (DNR),  which  can  provide  direction  for  standardizing  Fault 
Detection/Fault  Isolation,  and  Diagnostic  procedures  in  USAF  Data  Communi- 
cation Networks.  A basic  premise  of  this  concept  is  that  any  view  of  a 
clear  distinction  or  well  defined  line  of  demarcation  between  ADP  and  Data 
Communication  domains  is  an  artificial  one.  It  would  appear  that  any  sharp 
delineation  faded  some  time  ago  when  computer  design  passed  from  the  second 
generation  into  the  third  generation  architecture  by  introducing  the 
concept  of'  the  channel  controller.  This  statement  is  made  to  emphasize  the 
thought  that  fault  detection  and  diagnostic  procedures  must  include  all 
aspects  of  the  system  which  has  been  designed  to  formulate  and  transmit 
data  (no  boundaries). 

The  DNR  concept  employs  the  tools  of  both  hardware  and  software,  with 
signals  being  monitored  at  each  major  node  of  a network  by  a Nodal  Control 
Unit  (NCU).  These  NCUs  use  a microprocessor  to  monitor  either  the  data 
signals  being  transmitted  over  the  network  or  a set  of  standard  test 
signals  which  are  generated  by  the  Terminal  Interface  Units  (TIU).  The 
communication  system  operational  data  that  is  collected  by  the  NCUs  is 
analyzed  and  summarized  by  a Minicomputer  System  called  Oracle. 
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DATA  NETWORK  REEERRE  (DNR) 

CONCEPT  REPORT 

1.  BACKGROUND. 

1.1  INTRODUCTION.  This  technical  report  is  concerned  with  the  subject  of 
management  of  data  networks.  More  specifically,  it  is  concerned  with  the 
mechanics  of  data  network  monitoring,  fault  isolation,  and  network  diag- 
nostics . 

1.2  STATEMENT  OF  THE  PROBLEM  (GENERAL).  Just  as  man  himself  has  evolved 
to  his  present  level  of  development,  each  of  his  concepts,  his  products, 
and  his  natural  entrainment  into  integrated  endeavors  which  have  come  to  be 
known  as  "SYSTEMS",  appear  to  undergo  a similar  evolutionary  process.  The 
history  of  communications  systems  and  computer  Automatic  Data  Processing 
(ADP)  systems  attest  to  the  statement  made  above.  Data  processing  and  the 
requirement  for  data  transmission  lead  to  the  communication-computer  ADP 
complex.  Data  communication  networks  were  created  and  began  to  grow  in 
number  at  an  exponential  rate.  The  tools  to  manage  these  data  networks 
were  developed  at  a much  slower  rate  and  made  their  scarcity  known  to  the 
System  Engineer  by  widening  the  gap  between  that  which  is  designed  and 
desired,  and  that  which  is  achieved.  One  of  the  primary  reasons  for  this 
void  has  its  origin,  in  a natural  way,  in  the  evolutionary  flow  of 
technology.  The  fact  is  that  here  began  the  intersection  of  many  disci- 
plines which  up  to  this  time  had  been  separately  developed  each  with  its 
respective  terminology,  equipment,  and  standards.  This  scenario  is  not 
new;  to  the  contrary,  it  is  the  familiar  pattern  of  technological  develop- 
ment. 

As  data  networks  are  developed  and  expanded,  the  inverse  relationship 
between  network  complexity  and  system  reliability  becomes  much  more  evi- 
dent. As  the  network  grows  in  size,  failures  occur  more  frequently,  and 
the  period  of  time  necessary  for  diagnosis  and  remedial  action  (time  to 
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repair)  has  a tendency  to  increase.  Since  one  measure  of  system  downtime 
is  the  percentage  of  productive  time  lost  to  failures,  it  becomes  apparent 
that  downtime  increases  as  network  size  increases.  Downtime  is  reduced 
through  a concerted  effort  of  increasing  total  system  reliability  and 
providing  the  means  to  achieve  fast  and  accurate  fault  isolation  and 
remedial  action. 

The  computer  ADP  manager  learned  a long  time  ago  that  his  operations 
required  a set  of  systematic  procedures  and  diagnostics.  The  communica- 
tions engineer  had  established  his  working  practices,  tested  by  years  of 
experience  and  guided  by  the  demanding  dictums  of  spectral  purity  and  the 
limitations  of  bandwidth,  with  full  knowledge  of  the  fact  that,  "eternal 
vigilance  is  the  price  of  good  communications."  What  happens  then  during 
the  synergistic  mingling  of  these  disciples?  A rather  superficial  answer 
to  this  question  is,  of  course,  that  a new  entity,  a new  system  emerges. 
This  new  system  comes  equipped  with  its  own  set  of  characteristics,  its  own 
fxsrsonality , and  the  law  of  filial  regression  appears  to  make  itself  known 
in  the  inanimate  world.  Only  when  this  new  system  behaves  as  a recalci- 
trant offspring  do  we  begin  to  ask  ourselves  the  proper  questions. 

1.3  DATA  NETWORKS  AND  THE  U.S.  AIR  FORCE  (A  Survey  of  some  existing 
networks).  With  the  aim  of  establishing  some  bounds  on  the  data  network 
fault  isolation  and  diagnostic  problem,  as  it  exists  in  the  USAF,  a 
questionnaire  was  sent  out  to  various  commands.  Weather  nets,  command  and 
control  networks,  and  various  common-user  systems  were  included  in  this 
survey.  The  object  of  this  questionnaire  was  to  identify  the  magnitude  of 
the  problem,  and  gather  information  on  how  existing  network  management 
tools  and  functions  were  performing  their  jobs  of  fault  isolation  and 
diagnostics.  The  questionnaire  was  organized  to  determine  the  procedures, 
equipments  and  capabilities  possessed  by  a given  network.  (A  copy  of  the 
questionnaire  is  included  as  Appendix  A) . 
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1.3-1  Data  Network  Questionnaire . Questionnaires  were  sent  to  the 
management  of  twenty-one  data  networks.  Fifteen  of  these  twent.y-one 
questionnaires  were  returned.  A list  of  the  networks  returning  the 
questionnaire  is  given  in  Table  I.  One  of  the  first  requests  made  by  the 
questionnaire  had  to  do  with  the  user's  overall  evaluation  (in  terms  of 
satisfactory  or  unsatisfactory)  of  the  data  network  performance.  It  is 
interesting  to  note  that  of  the  fifteen  networks  responding  to  the 
questionnaire,  only  two  reported  an  overall  performance  evaluation  of 
unsatisfactory . 

The  questionnaire  reflects  a wide  variety  of  network  configurations, 
equipments  and  operation  procedures.  These  networks  vary  in  size  and  have 
been  engineered  to  use  various  combinations  of  bit  rates,  synchronization 
techniques,  and  multiplex  schemes.  Their  topologies  range  from  point-to- 
point  to  full  fledged  communication  data  networks.  Data  rates  of  these 
networks  were  classified  as  low  speed  (0-1200  bps),  medium  speed  (1200-^1800 
bps),  and  high  speed  (4.8-19-2  Kbps).  Of  the  fifteen  responses,  over  50% 
of  the  networks  had  multiplexed  circuits.  Four  of  these  systems  operate  at 
data  rates  of  9b00  bps  or  greater,  with  two  systems  going  up  to  19-2  Kbps. 
As  a measure  of  the  fault  isolation  and  diagnostic  capabilities,  several 
questions  were  asked  concerning  fault  detection,  diagnostic  equipment, 
data  network  [performance  history,  and  trouble-shooting  procedures.  Of 
those  answering,  only  one  had  all  the  capabilities,  and  only  one,  again, 
had  none  of  the  capabilities.  Diagnostic  procedures  appears  to  be  the  most 
lacking.  Most  of  the  responses  indicated  that  the  network  troubleshooters 
worked  on  the  basis  of  "experience",  and  they  had  no  set  course  of  arriving 
at  the  cause  of  the  problems. 

1-3-2  Domains  and  Parameters.  In  determining  the  equipments  (test  and 
measurement)  to  be  used  for  fault  isolation  and  diagnostics,  it  is  conve- 
nient to  think  of  the  network  as  being  divided  into  three  domains:  the 
data,  time  and  frequency  domains.  See  Fig  1.1.  Each  domain  has  a set  of 
parameters  which  characterize  it  and  which  can  be  monitored. 


TABLE  I 

DATA  NETWORK  SYSTEMS 


Users  Overall  Evaluation  of  Data  Network  Performance 


ADP-COMMUNICATION  NETWORK  DESIGNATOR  S 


1.  AFLC  Responsive  Communications  Network  (ARC)  X 

2.  Computer  Resources  For  Engineering  Simulation, 

Training,  Education  (CREATE)  WPAFB,  ACDSM, 

DPI  5802  (AFLC)  X 

3.  Maintenance  Analysis  & Structural 

Integrity  Information  System 
(MASIIS)  (AFLC) 

4.  Numeric  Control  (NUMCON)  (AFLC)  X 

5.  Automatic  Test  Equipment  (AFLC)  X 

6.  Management  Information  & Text  System  (MITS) 

Hq.  Air  Force  Audit  Agency  X 

7.  Major  Command  ADP  Update  System  (ADCOM  MAJCOM)  X 

8.  Air  Force  Accounting  St  Finance  Center 

Teleprocessing  (AFAFC  Teleprocessing)  X 

9.  Military  Airlift  Command  Integrated  Management 

System  (MACIMS)  (MAC) 

10.  Strategic  Air  Command  Total  Information 

Network  (SATIN)  (SAC) 

11.  Rome  Air  Development  Center  System  (RADC)(AFSC)  X 

12.  Information  Central/Acquisition  Management 

Information  System  ( INFOCEN/AMIS) (AFSC)  X 

13.  Air  Force  Systems  Command  Network  (AFSC  NET)  X 

14.  Automated  Weather  Network  (AWN)  (CCPC)  X 

15.  AFCS  MAJCOM  Update  Computer  Network  X 


UnS  UnK* 


X 


X 


X 


* S=Sa t is factory/UnS= Unsat is factory/UnK= Unknown 
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Figure  1.1  Domains 


1.3.2. 1  The  Data  Domain.  The  data  domain  is  defined  to  be  that  area  where 


the  process  of  information  transfer  starts.  It  is  here  the  bits  of 
information  are  collected  and  arranged  for  transfer  through  the  system. 
Protocol  and  control  functions  such  as  parity  assignment  and  checking  are 
performed.  Software  diagnostics  of  the  data  and  of  the  data  communications 
system  are  applied.  These  operations  can  take  place,  for  example,  in  such 
devices  as  computers,  communications  processors,  and  terminals.  Test  and 
monitor  procedures  can  reside  in  software  and/or  hardware.  The  Spectrum 
Datascope , Halcyon,  and  Intershake  diagnostic  equipment  are  examples  of 
data  domain  tools  which  allow  the  troubleshooter  to  look  at  the  control 
characters  as  well  as  the  data.  This  type  of  diagnostic  tool  is  valuable 
because  -t  permits  software  problems,  such  as  line  protocol  to  be  observed. 
Only  five  of  the  systems  responding  to  the  questionnaire  had  such  equip- 
ment. 

1.3 .2. 2 The  Time  Domain.  In  the  time  domain,  the  information  bits  have 
passed  from  the  computer  or  terminal  to  the  interface  at  the  modem, 
multiplexer  or  other  data  transmission  device.  Parameters  which  charac- 
terize this  domain  are  clock  slips,  skew,  distortion,  intersyrabol  inter- 
ference and  jitter  associated  with  the  modem  recovery  system.  The  presence 
of  such  phenomena  indicates  the  kind  of  problems  associated  with  improperly 
adjusted  thresholds  and  possible  failure  in  the  synchronization  circuitry. 
Of  course,  clock  slips  can  result  from  phase  bits  on  the  channel.  Test 
equipment  for  this  domain  consists  of  such  equipment  as  bit  error  rate 
checkers  and  oscilloscopes.  Seven  of  the  data  systems  responding  indicated 
the  use  of  such  equipment. 

1.3 .2. 3 The  Frequency  Domain.  The  frequency  domain  of  the  data  communica- 
tions system  is  that  area  in  which  the  data  signals  are  in  analog  form.  The 
parameters  are  those  which  characterize  the  voice  frequency  (V.F.)  line. 
These  parameters  include,  for  example,  amplitude  distortion,  envelope 
delay  distortion,  signal  to  noise  ratio  and  phase  jitter.  A discussion  of 
the  parameters  pertaining  to  the  V.F.  line  is  contained  in  Appendix  B of 
this  report.  Much  of  the  frequency  domain  equipment/capabilities  used  for 
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testing  has  centered  around  signal  comparison  and  loopback  techniques.  An 
excellent  technique  for  this  area  is  to  monitor  the  eye-pattern  of  the  V.F. 
circuit.  Only  four  of  the  systems  responding  possessed  capabilities  in 
this  area.  This  domain  appeared  to  be  the  most  lacking  in  capabilities. 

l.ii  SOME  CONCLUDING  REMARKS  ON  THE  QUESTIONNAIRES.  Table  II  summarizes 
network  diagnostic  capabilities  of  the  data  systems  answering  the 
questionnaire.  To  this  information  must  be  added  that  all  of  the  systems 
reporting  were  dealing  with  at  least  two  vendors,  with  some  as  many  as  six 
vendors.  The  orchestrations  for  "finger-pointing"  are  obvious.  Such 
statements  as  "not  knowing  which  vendor  to  call",  and  "vendors  not  assuming 
responsibility  for  an  outage"  w^re  common.  In  addition,  six  of  the  systems 
had  some  form  of  encryption. 

The  requirement  for  an  organized  approach  to  the  fault  isolation  and 
diagnostic  problem  is  evident.  Additional  capability  in  the  forms  of 
hardware  and  software  can  be  used  by  existing  data  systems,  and  a great 
deal  of  planning  is  needed  for  any  future  systems  yet  to  be  considered. 

Although  the  questionnaires  report  trouble  to  some  extent  in  every  facet  of 
the  communication  system,  it  appears  that  in  general,  networks  that  cover  a 
large  geographical  area  (CONUS  wide)  devote  a preponderance  of  their 
trouble-shooting  time  to  problems  in  the  communication  lines;  whereas, 
networks  that  cover  a relatively  small  area,  such  as  a base,  spend  the 
major  portion  cf  their  trouble-shooting  time  on  communication  equipment 
and  terminals. 

The  dependence  of  data  transmission  error  on  transmission  rate  is  again 
shown  in  the  responses,  with  the  reported  error  rates  increasing  as  the 
transmission  rate  goes  up.  Also  reported,  as  would  be  expected,  was 
increa.sed  downtime  for  systems  using  older  equipment  (reliability  factor), 
and  protocol  problems  associated  with  new  systems. 
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TABLK  ir 


SOME  NUMBERS  FROM  THE  QUESTIONNAIRES 


Number  of 

Networks 

Reporting 

CAPABILITY Capability 

Systems  with  Fault  Detection  9 

Systems  with  Diagnostic  Equipment/Capabilities  9 

Systems  with  Diagnostic  Equipment/Capabilities  as 

Per  Domain 

Data  Domain  5 

Time  Domain  7 

Frequency  Domain  ^ 

Systems  with  Performance  History  6 

Systems  with  Troubleshooting  Procedures  3 

Those  Requesting  Additional  Fault  Isolation  and 
Diagnostic  Capabilities 

In  Terms  of  Hardware  1 1 

In  Terms  of  Software  7 
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Finally,  lack  of  available  equipment  spares,  vendor  response  time  measured 
in  terms  of  hours,  little  or  no  systematic  procedures  for  attacking  an 
existing  problem  (in  some  cases  no  detection  of  an  existing  problem),  no 
capability  for  alternate  routing,  and  a scarcity  of  record  keeping  on 
system  outages  and  how  they  were  corrected,  all  add  to  the  general 
commentary  on  the  status  quo  of  many  existing  data  communication  systems. 

2.  DATA  NETWORK  REFEREE  (DNR) 

2.1  INTRODUCTION.  This  section  will  discuss  a concept  of  network  monitor- 
ing which  has  been  given  the  name  of  Data  Network  Referee  (DNR).  The 
purpose  of  the  DNR  is  to  perform  fault  isolation  and  diagnostics  of  data 
communication  networks.  The  objectives  of  the  DNR  are  discussed  in  the 
following  paragraphs. 

2.2  OBJECTIVES  OF  DATA  NETWORK  REFEREE  CONCEPT. 

2.2.1  Assign  Responsibility.  The  DNR  subsystem  must  assign  responsibility 
for  data  communication  system  failure.  It  must  be  able  to  rapidly  detect  a 
fault  in  any  area  of  the  data  communication  system  and  identify  the  failed 
function . 

2.2.2  Enhance  Availability.  The  DNR  subsystem  must  enhance  the  data 
communication  system  availability  by  decreasing  system  down  time  (increase 
Reliability).  This  is  done  by: 

a.  Providing  a set  of  systematic  test  and  record  keeping  techniques. 

b.  Providing  a means  to  identify  potential  trouble  areas,  and  making 
available  the  control  functions  to  introduce  communication  system 
operating  options  where  they  are  available  and  applicable. 
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2.2.3  Feasibility.  The  DNR  subsystem  must  be  feasible  both  technically 
and  economically.  The  system  will  utilize  proven  STATE-OF-THE-ART  com- 
ponents and  subsystems,  and  should  be  easy  to  use  and  not  require  an 
engineer  to  operate. 

2.2.4  Compatibility.  The  DNR  subsystem  must  be  compatible  with: 

a.  Common  Carrier  equipment  and  policy. 

b.  Established  network  practices  and  protocols. 

3.  THE  APPROACH  TO  THE  CONCEPT. 

I 

3.1  WHAT  THE  SYSTEM  DOES.  It  was  decided  of  necessity  that  a useful 
network  monitoring  system  must  accomplish  a certain  minimum  number  of 
functions.  First,  it  must  be  able  to  monitor  the  performance  of  the 
network  while  data  is  being  passed.  Secondly,  it  must  be  able  to  initiate 
and  perform  tests  on  the  network,  not  only  when  a circuit  fails,  but 
periodically  as  a preventative  maintenance  measure  and  as  a means  of 
determining  system  availability.  It  is  a must  that  such  a system  be  able 
to  accurately  and  rapidly  diagnose  problems  to  an  equipment  level  (modem, 
multiplexer,  line,  etc.)  so  as  to  permit  diagnosis  and  eventual  restora- 
tion. The  system  must  have  a means  of  reporting  results  to  one  or  more 
operators  via  printed  output  and  major/minor  alarms.  The  system  must  also 
have  a means  of  collecting  and  recording  data  in  real  time  both  as  an  aid 
I for  fault  diagnosis  and  for  keeping  an  account  of  system  status  over  a 

j period  of  time.  Finally,  a capability  to  analyze  the  recorded  data  over  a 

[ period  of  time  (non-real  time)  is  required.  Such  a capability  would  be 

i valuable  for  correcting  persistent  faults  and  predicting  future  system 

'i 

availability  and  performance  requirements. 
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The  concept  was  first  formulated  by  analyzing  the  basic  anatomy  of  a data 
communications  network.  Typically  such  a network  consists  of  a data 
center,  such  as  one  or  more  CPUs,  with  many  data  links  radiating  outward, 
like  the  spokes  of  a wheel,  to  terminals  or  other  computers  at  remote 
locations.  At  various  places,  each  of  these  links  (comprising  many 
channels)  may  further  branch  out  to  form  several  individual  links.  These 
points  of  branching  or  concentration  are  called  "nodes"  and  are  usually 
characterized  by  the  location  of  one  or  more  multiplexers,  modems,  and 
other  equipment  at  that  point.  Therefore,  the  basic  data  network  anatomy 
can  be  represented  by  major  links,  nodes,  and  branches  as  shown  in  the 
generic  example  of  Figure  3.1. 

This  fixed  topological  structure  gives  rise  to  the  basic  idea  of  monitoring 
the  performance  of  the  network  at  every  node  (with  a "Nodal  Control  Unit" 
or  "NCU")  and  at  every  entrance  into  the  network  communication  system 
(through  a "Terminal  Interface  Unit"  or  "TIU")  which  is  located  between 
every  terminal  (or  computer)  and  the  input  to  the  communication  line  which 
is  usually  a modem  or  line  driver.  Thus  we  have  the  basic  concept 
requirement  that  data  from  a terminal  or  computer  must  first  pass  through  a 
TIU  before  entering  the  communications  network  and  every  node  of  the 
network  must  be  monitored  by  an  NCU.  As  required,  NCUs  within  a data 
network  will  be  able  to  communicate  with  each  other. 

The  final  aspect  of  the  concept  involves  a third  element  called  the  ORACLE 
whose  main  function  is  to  gather  the  recorded  data  from  all  NCUs  in  the 
system  and  analyze  it  over  a period  of  time.  Thus  ORACLE  from  time  to  time 
is  able  to  report  on  the  overall  condition  (status)  of  the  data  communica- 
tion networks  it  monitors.  Types  of  analyses  outputted  would  be  statisti- 
cal in  nature  and  would  include  past  performance,  testing  procedures  used, 
areas  of  weakness,  and  predictions  on  future  system  performance  and 
availability.  If  communication  between  the  NCUs  and  ORACLE  were  lost,  the 
NCUs  and  TIUs  would  be  capable  of  performing  the  necessary  fault  isolation 
and  diagnostics  but  with  reduced  long  term  effectiveness.  Figure  3.2 
illustrates  the  basic  struture  of  the  concept. 
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Figure  3.1  TOPOLOGY  OF  A TYPICAL  DATA  COMMUNICATIONS  NETWORK 
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4.  HOW  THE  SYSTEM  FUNCTIONS. 


4.1  GENERAL.  An  example  of  a typical  data  link  is  shown  in  Figure  4.1. 
Ideally,  the  system  will  employ  modems  capable  of  self  testing  with  output 
ports  to  monitor  the  results.  Typically  the  status  lamps  output  of  a high 
speed  modem  (take  for  example  a modem  operating  at  9600  bps)  consists  of 
three  lights  indicating  good,  marginal  and  bad  data  conditions.  These 
conditions  and  others  will  be  automatically  monitored  and  input  to  the  NCU 
at  the  node.  Also  these  modems  ideally,  when  turned  on,  go  into  a training 
mode  with  a modem  at  the  opposite  end  of  the  line  for  the  purposes  of  line 
equalization.  Likewise  the  multiplexer  will  be  monitored  directly  for 
correct  frame  and  bit  timing.  Hence,  faults  in  either  of  these  equipments 
can  be  monitored  directly  by  the  NCU  located  at  that  location. 

The  individual  communication  circuits  between  the  multiplexer  and  the 
terminals  pass  through  a patch  panel  and  bi-directional  scanner  (BDS).  The 
BDS  allows  the  NCU  to  address  any  line  and  either  monitor  it  or  patch  into 
it.  Thus  any  one  line  (or  all  serially)  can  be  monitored  while  data  passes 
through  it  or  a signal  can  be  sent  or  received  on  the  line  in  either 
direction  by  the  NCU.  The  individual  lines  from  the  node  may  travel  a 
relatively  short  distance  to  a nearby  terminal,  or  they  may  go  to  a remote 
location  (across  base  for  example)  in  which  case  low  speed  modems  may  be 
employed.  One  of  the  modems  for  this  circuit  would  be  located  at  the  node, 
and  if  it  wsre  not  of  the  type  that  had  a self  testing  capability,  it  would 
be  retrofitted  so  that  either  its  performance  at  the  NCU  could  be  monitored 
directly  or  the  NCU  could  cause  it  to  switch  in  a loopback  mode  (by  a direct 
connection).  Such  an  arrangement  is  illustrated  in  Figure  4.2a  and  4.2b. 

Again  referring  to  Figure  4.1,  a TIU  is  located  between  the  terminal  and 
its  entrance  to  the  communications  line  (which  may  be  a modem  or  a line 
driver  as  the  case  may  be).  The  TIU  will,  in  addition  to  sending  and 


Figure  A.  2a  NCU  INITIATING  LOOPBACK  TEST  ON  LOW  SPEED  MODEM 


Figure  4.2b  NCU  DIRECTLY  MONITORING  MODEM  PERFORMANCE 
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receiving  test  signals,  be  able  to  initially  switch  the  modem  or  line 
driver  at  the  terminal  into  a loopback  mode  by  sending  a test  signal  to  it 
and  evaluating  its  return.  This  simple  test,  illustrated  in  Figure  4.3, 
can  be  performed  when  the  terminal  is  initially  switched  on  and  off. 

One  of  the  fundamental  requirements  of  this  concept  is  that  the  terminal's 
on/standby  switch  control  the  operational  mode  (data/test)  of  the  TIU. 
When  the  terminal  is  on  and  sending  or  receiving,  the  data  passes  through 
the  TIU  unhindered.  When  the  terminal  is  switched  to  standby  mode,  the  TIU 
is  automatically  activated  (by  relay  or  other  means)  to  its  test  mode  and  a 
fixed  test  pattern  sent  out  on  the  line.  Thus  the  network  is  constantly 
being  exercised  either  in  the  data  or  the  test  mode.  Lack  of  any  signal  is 
an  indication  of  a circuit  fault. 

4.2  CIRCUIT  MONITORING  AND  TESTING.  Looking  at  the  data  circuit  in 
Figure  4.1,  during  the  data  mode,  the  NCUs  are  systematically  scanning  each 
line  and  monitoring  various  aspects  of  the  data  (timing,  etc.)  and  con- 
stantly comparing  it  with  stored  preset  standards.  Irregularities  are 
alarmed  and  recorded  and,  if  necessary  a test  is  initiated  on  the  partic- 
ular line.  In  addition,  the  NCJs  are  directly  monitoring  the  modems  and 
multiplexers  at  the  nodes.  Thu  following  parameters  are  analyzed  by  the 
NCU  while  data  is  being  passed: 

Synchronization 

Receive  clock 
Send  Clock 

Frame  synch  (Multiplexer) 

Delay  time 
Spectrum  Ratio 
Continuity  of  data  signal 
Carrier  loss  (modem) 

On-Line  Indication  of  Line  Quality  (Eye-Pattern) 

Quality  of  data  (modem) 
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Figure  4.3.  TIU  INITIATING  LOOPBACK  TEST  ON  LOW  SPEED  MODEM 
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Let  us  assume,  at  this  time,  a 4-wire  (full  duplex)  circuit.  When  the 
terminal  is  placed  on  standby,  the  TIU  immediately  checks  out  "its  modem" 
(or  line  driver)  by  performing  a loopback  test.  A faulty  modem  triggers  an 
alarm  (which  may  be  a light)  at  the  TIU  where  the  terminal  operator  can 
recognize  that  the  modem  is  faulty.  The  operator  may  then  report  this 
condition  to  an  operator  at  one  of  the  NCUs  by  a phone  call.  If  all  is 
well,  the  TIU  puts  out  a fixed  test  pattern  (call  it  "A")  on  the  line.  If 
the  system  is  operating  properly,  the  "A"  reaches  the  first  NCU  (NCU-1) 
which  recognizes  it  and  immediately  goes  from  the  monitor  mode  to  the  patch 
mode  and  resends  the  test  signal  "A"  down  the  line  to  the  next  NCU  (NCU-2) 
which  recognizes  it  as  the  valid  test  pattern.  NCU-2  also  goes  into  the 
patch  mode  and  sends  "A"  down  the  line  to  TIU— 2 at  the  network  center. 
(There  could  be  other  NCUs  along  the  way  depending  on  the  number  of  modems 
along  the  line.) 

TIU-2  recognizes  the  "A"  signal  and  goes  into  the  test  mode,  isolating  the 
CPU  from  the  line  and  sending  a "B"  test  pattern  back  to  NCU-2  which  in  turn 
recognizes  the  "B"  and  resends  it  to  NCU-1.  NCU-1,  recognizing  B,  resends 
it  to  TIU-1.  When  TIU-1  sees  the  "B"  pattern,  it  will  continue  sending  the 
"A"  to  NCU-1.  Failure  to  receive  the  proper  test  pattern  at  the  NCUs  and 
TIUs  causes  an  alarm  and  printout  to  occur.  By  knowing  at  what  points  the 
signal  was  good  and  bad,  coupled  with  direct  performance  of  the  modems  and 
multiplexers,  allows  the  faulty  portion  of  the  circuit  to  be  isolated. 

If  TIU-2  at  the  network  center  (CPU)  fails  to  receive  the  correct  "A" 
pattern,  it  will  continue  to  operate  in  the  data  mode  not  being  aware  that 
tne  remainder  of  the  network  is  in  the  test  mode.  Failure  of  NCU-2  (which 
is  in  the  test  mode)  to  receive  a "B"  from  TIU-2  causes  an  error  to  be 
flagged  for  that  portion  of  the  circuit.  In  any  case,  NCU-2  will  send  the 
correct  "B"  pattern  to  NCU-1  to  check  that  portion  of  the  circuit.  NCU-1 
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will  resend  the  "B"  to  TIU-1  which,  upon  recognizing  it,  will  continue 
sending  "A"  to  NCU-1.  If  TlU-'i  does  not  receive  a "B",  it  will  send  a fault 
pattern  (F^)  to  NCU-1.  It  should  be  noted  that  in  the  event  that  the 
circuit  between  TIU-1  and  NCU-1  is  faulty  (for  example  a break  in  the  line 
at  point  "X"  in  Figure  4,1),  the  "A"  test  pattern  will  never  reach  NCU-1, 
and  the  rest  of  the  system  will  never  know  TIU-1  is  in  the  test  mode.  This 
problem  is  overcome  since  these  "unrecognizable"  signals  will  eventually 
reach  the  CPU  causing  "NACKS"  to  be  constantly  returned.  These  can  be 
recognized  by  NCU— 2 which  would  then  place  the  system  in  a test  mode  to 
isolate  the  problem.  In  addition,  the  operator  at  the  terminal  can  call  an 
operator  at  one  of  the  NCUs  (or  vice-versa)  and  report  erroneous  data.  The 
terminal  and  NCU  operators  can  then  place  the  circuit  in  the  test  mode  from 
both  ends  and  quickly  isolate  the  problem. 

When  the  system  is  in  the  test  mode  the  NCUs  check  the  fixed  test  patterns 
for  the  following  by  comparing  them  with  stored  replicas: 

Bit  Error  Rate  (BER) 

Phase  error 
Phase  jitter 
Phase  hits  and  dropouts 
Impulse  noise  (Gain  hits) 

Pulse  level 
Clock  Slip  Rate 

One  of  the  characteristic  features  of  this  concept  is  that  all  monitoring 
and  testing  of  the  lines  is  done  on  the  digital  portion  only.  This 
minimizes  the  necessity  of  expensive  analog  measuring  equipment.  Further- 
more, the  above  digital  test  should  prove  sufficient  for  isolating  and 
diagnosing  most  of  the  common  faults  in  a system.  A more  detailed  analysis 
of  the  circuits  can  be  made  with  more  sophisticated  portable  equipment  at 
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the  patch  panels  that  would  exist  along  the  line.  Such  would  probably  be 
the  case  for  quality  of  service  acceptance  tests. 

It  is  the  intention  of  this  concept  that  the  network  monitoring  system  be 
as  flexible  as  possible  and  take  advantage  of  existing  facilities.  For 
example,  if  a spare  channel  exists  between  two  nodes  (one  leg  of  a 
multiplexer),  it  may  le  used  by  the  NCUs  to  pass  a test  code  pattern 
(pseudorandom  or  fixed)  so  as  to  monitor  that  portion  of  the  network 
continually.  In  the  case  of  two-wire  (half  duplex)  lines,  the  test  pattern 
sequence  would  have  to  be  modified;  however,  the  basic  idea  would  remain 
the  same . 

NODAL  CONTROL  UNIT  (NCU).  The  NCU  is  the  most  important  element  in  the 
network  monitoring  system.  It  performs  the  following  functions: 

a.  Monitors  each  circuit,  modem,  multiplexer,  crypto  unit  or 
other  such  equipment  at  a node. 

b.  Initiates  tests  on  each  circuit. 

c.  Collects  and  records  data  on  system  performance  in  real  time. 

d.  Prints  out  information. 

e.  Provides  alarms. 

f.  Performs  a self-test  to  insure  accurate  operation. 

g.  Communicates  with  other  NCUs  and  ORACLE  as  required  on  a dial 
up  basis. 
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Figure  shows  the  basic  components  required  in  the  NCU.  Except  for  the 
I/O  devices  such  as  the  printer  and  keyboard,  the  power  required  would  not 
be  very  great.  The  possibility  then  exists  for  use  of  a battery  (emer- 
gency) supply  which  could  be  kept  on  float  during  operation  from  external 
power.  Also,  the  keyboard  and  printer  could  be  plug-in  (optional)  in  the 
case  of  NCUs  located  at  nodes  distant  from  the  network  center. 

A printout  from  the  NCU  would  consist  of  the  status  of  the  system  upon  the 
conclusion  of  any  test  or  upon  detecting  any  deviation  from  normal  opera- 
tion. The  printout  would  include  a unit  (modem,  etc)  or  line  identifier 
followed  by  the  detected  condition.  It  would  be  optional  to  print  out  if  a 
unit  or  line  were  operating  normally.  Each  system  would  have  a plan 
showing  each  unit  and  line  with  its  identification  (alphanumeric  or 
otherwise)  clearly  noted.  This  would  keep  the  microprocessor  language 
simple  and  save  storage  space.  The  date  and  time  of  each  incident  would  be 
noted.  An  example  of  a printout  might  look  like  this: 

01  June  77  1330 
M0D1:CD,  MUX2:05,  L38:SL 
L38:BER 

01  Jun  77  1^130 
L38:0K 

The  above  printout  says  that  on  1 June  77  at  1330  hours,  "Modem  1"  had  a 
carrier  dropout,  multiplexer  2 went  out  of  synch,  line  38  had  a signal 
loss,  and  line  38  failed  the  BER  criteria  on  the  test  pattern.  At  1il30 
hours  the  same  day,  line  38  was  restored  to  an  acceptable  condition.  The 
operator  at  the  NCU  would  refer  to  a manual  of  the  system  to  find  the 
location  of  a particular  unit  or  line.  The  above  merely  indicates  what 
might  consist  as  a printout.  Trial  and  error  on  a concept  implemented 
system  would  resolve  the  best  format  for  data  printout. 
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I Figure  4.4  NODAL  CONTROL  UNIT  (NCU) 
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4.4  TERMINAL  INTERFACE  UNIT  (TIU).  The  TIU  is  the  entry  point  into  the 
data  communication  system.  It  performs  the  following  basic  functions: 

a.  Acts  as  the  interface  between  terminals  and  the  communication 
system. 

b.  Sends  and  receives  test  patterns. 

c.  Performs  tests  on  associated  modems  or  other  equipments  (line 
drivers,  etc.) 

d.  Performs  a self-test  to  insure  accurate  operation. 

Physically,  the  TIU  can  be  configured  on  an  electronic  card  and  installed 
either  in  the  terminal  (where  new  systems  are  being  considered)  or  in  a 
separate  small  box  and  located  adjacent  to  the  terminal,  in  the  case  of 
existing  systems. 

Figure  4.5  shows  the  basic  components  required  in  the  TIU.  Note  that  the 
TIUs  located  at  the  network  center  (CPU)  do  not  require  a fault  pattern 
transmitter  as  do  the  TIUs  at  the  terminals.  Figure  4.6  shows  the  controls 
required  at  the  TIU  to  assist  the  terminal  operator  in  ascertaining  the 
status  of  the  network  monitoring  system  at  his  end.  When  the  Auto/Manual 
Test  switch  is  in  the  "Auto"  position,  the  TIU  mode  is  controlled  by  the 
On/Standby  (or  off)  switch  on  the  terminal.  The  "manual"  position  places 
the  TIU  in  a constant  (manual)  test  mode.  Indicator  lights  show  whether 
the  TIU  is  in  the  test  or  data  mode.  When  the  Modem  Test  switch  is  in  the 
"Auto"  position,  the  modem  is  checked  when  the  system  is  initially  placed 
in  the  test  mode.  The  "Manual"  position  places  the  modem  in  test  all  the 
time  (manually).  Indicators  show  when  the  modem  is  being  tested  and  if  the 
modem  is  faulty.  There  is  an  indicator  to  show  if  the  TIU  self-test  fails 
(TIU  is  not  functioning  properly)  and  if  a crypto  unit  associated  with  the 
terminal  has  reset.  (See  Section  for  the  discussion  of  a crypto  associated 
circuit. ) 
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Figure  4.5  TERMINAL  INTERFACE 
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^.5  SYSTEM  OPERATION  WITH  CRYPTO  UNITS.  If  the  communication  network 
employs  encryption  devices  (either  single  channel  or  bulk),  the  situation 
is  a bit  more  complicated.  In  the  case  of  single  channel  encryption,  the 
solution  is  to  place  the  TIUs  after  the  crypto  units  as  illustrated  in 
Figure  4,7.  If  this  were  not  so,  the  NCUs  would  not  recognize  the  test 
pattern  sent  by  the  TIUs.  Any  monitoring  lines  from  crypto  units  would 
have  to  be  isolated  before  going  to  the  TIU.  If  a terminal  is  located  in  a 
room  separate  from  the  crypto  unit  and  modem,  an  isolated  cable  would  be 
run  to  the  terminal  location  to  a remote  unit  having  the  TIU  switch  and 
indicator  functions. 

If  bulk  encryption  is  used,  the  situation  is  easier  since  the  encryption 
device  is  located  by  the  multiplexers  as  shown  in  Figure  4.8.  In  this 
case,  the  NCU  will  be  evaluating  "red"  data  and  would  be  located  in  a 
secure  area.  The  NCU  would  have  to  be  properly  isolated  when  communicating 
with  another  NCU  in  a "black"  area  or  ORACLE  which  would  definitely  be  in  a 
"black"  area.  This  situation  is  discussed  in  the  section  detailing  the 
description  and  function  of  ORACLE. 


4.6  ORACLE.  One  of  the  basics  of  the  network  monitoring  concept  is  ORACLE 
which  is  envisioned  as  a central  processing  facility  which  receives  inputs 
from  the  NCUs  of  the  various  subscribing  data  networks  and  performs  the 
following  functions: 

a.  Collects  data  from  NCUs. 

b.  Checks  and  collates  this  data. 

c.  Performs  analyses  (statistical) 

Short/long  term  trends 
System  error  analyses 
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Figure  4.7  SINGLE  CHANNEL  ENCRYPTION  CONFIGURATION 


d.  Initiates  test  procedures  (on  data  system  and  NCUs) 
and  records  test-fault-remedial  action  sequences. 


e.  Updates  system  records  and  files. 

f.  Prints  out  periodic  reports. 

4.7  Oracle  Configurations.  ORACLE  can  assume  a number  of  configurations, 
only  two  of  which  will  be  discussed  here. 

CONFIGURATION  I:  ORACLE  assumes  the  form  of  an  independent  minicomputer 
Data  Base  System  which  operates  on  DNR  data  received  from  the  NCUs  of  a 
data  communication  network.  In  this  configuration,  ORACLE  is  not 

necessarily  dedicated  to  one  data  communication  network.  It  can  serve  a 
number  of  data  networks  by  allocating  a fixed  amount  of  its  time  and  assets 
to  handle  preselected  routines  for  each  data  communication  network  that 
joins  the  system,  while  keeping  in  reserve  some  of  its  capabilities  for 
special  functions  and  situations  that  will  arise  in  a given  member  system. 

Data  comes  to  ORACLE  over  DIAL-UP  lines.  Four  cases  are  considered: 

a.  ORACLE  calls  the  NCU  for  Data  Dump. 

In  this  case,  the  data  that  is  recorded  in  the  tape  cassette  of  the  NCU  of  a 
DNR  subsystem  is  sent  to  ORACLE.  This  data  contains  the  results  of  the 
major  activities  and  deviations  from  certain  prescribed  criteria  at  that 
node  over  some  period  of  time.  The  duration  of  time  covered  by  a given 
"Cassette  Dump"  could  be  a day  or  a week  or  whatever  duration  of  time  that 
proves  satisfactory  for  successful  operation  of  the  system.  ORACLE 
receives  this  data  by  calling  the  NCUs  on  the  DIAL-UP  line  and  requesting  a 
dump.  This  is  a routine  operation  and  initiated  under  the  routine  Data 
Collection  Mode  of  ORACLE. 
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b.  NCU  calls  ORACLE  for  Assistance.  This  case  will  usually  be 
initiated  by  the  NCU  while  in  an  emergency  situation.  This  can  happen,  for 
instance,  when  major  performance  criteria  have  been  failed  or  exceeded 
certain  acceptable  margins  at  a particular  NCU.  The  particular  NCU  at 
which  this  condition  has  occurred  may  be  unattended,  and  the  NCU  calls 
ORACLE  or  the  Master  NCU  of  the  system.  This  is  a cry  for  "help"  from  the 
NCU  to  ORACLE.  In  this  case,  the  NCU  Dials-Up  ORACLE  and  receives 
immediate  attention.  When  action  has  been  taken,  the  communication  line  is 
dropped  by  ORACLE. 

c.  An  NCU  calls  another  NCU  through  ORACLE.  All  NCUs  have  the 
capability  of  alarming  and  printing  out  failure  conditions;  however,  not 
all  the  NCUs  will  require  attendants  or  an  operator.  If  an  alarm  condition 
exists  at  such  an  unsupervi.sed  NCU,  the  NCU  is  instructed  by  its  internal 
program  to  initiate  a call  to  a Master  NCU  whose  outputs  are  supervised  or 
monitored  by  operating  personnel.  This  call  could  go  through  ORACLE  or 
direct  to  the  Master  NCU.  With  this  call,  whatever  alarm  conditions  exist 
at  the  troubled  NCU  will  be  alarmed  and  printed  out  at  the  Master  NCU. 

d.  ORACLE  calls  one  or  more  NCUs  to  instruct.  This  case  can  be 
thought  of  as  a Command  Call  made  by  ORACLE  to  one  or  more  NCUs  in  a data 
communication  system.  The  purpose  of  this  call  is  to  place  in  the  NCU  some 
instruction  or  special  test  to  be  performed  at  that  node.  For  instance,  it 
could  command  the  NCU  located  at  point  A to  load  and  transmit  a pseudo- 
random sequence  or  "Fox  Message"  to  an  NCU  located  at  point  B and  have 
NCU-B  make  the  applicable  tests  on  these  signals. 

CONFIGURATION  II:  Another,  more  conservative,  approach  to  ORACLE  is  to 
have  its  functions  accomplished  by  the  computer  associated  with  the  data 
communication  network.  This  arrangement  is  feasible  only  when  the  ADP 
system  has  room  to  spare,  a possibility  that  is  usually  rather  remote.  The 
main  reason  for  its  discussion  here  is  that  it  represents  one  way  of 
starting  an  ORACLE  and  testing  its  feasibility  and  potential. 
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^.8  Existing  vs  Future  Data  Networks.  The  discussion  in  this  report 
concerned  itself  primarily  with  existing  data  communication  networks 
(along  with  the  myriads  of  different  modems,  multiplexers,  line  drivers, 
etc.)  and  a means  to  address  most  of  the  configurations  possible  with  these 
equipments.  Many  of  these  equipments  (for  example  low  speed  modems)  do  not 
incorporate  self-test  features.  Equipments  for  future  networks  along  with 
replacements  in  existing  networks  are  envisioned  as  containing  self- 
testing  features.  The  functions  of  the  TIU  could  be  built  into  the 
terminal.  Modems  would  perform  various  loopback  operations,  for  example, 
with  connections  for  external  control  and  monitoring.  This  would  simplify 
the  circuitry  of  the  NCUs  and  Tills  and  provide  for  a more  efficient  fault 
isolation  and  diagnostic  capability,  especially  in  the  analog  portions  of 
the  system.  The  purchase  of  such  equipments  capable  of  self-testing  (etc) 
should  be  incorporated  into  standards  for  communications  systems.  This  is 
further  discussed  in  the  Recommendations  portion  of  this  report. 

4.9  SYSTEM  RESTORATION  BY  NCUs.  The  previous  paragraphs  have  discussed 
the  DNR  concept  for  Fault  Detection  and  the  use  of  Diagnostics  for 
identification  of  the  system  function  that  has  failed.  If  the  DNR  is 
associated  with  a data  communication  network  that  employs  any  redundancy  in 
its  communication  subsystems,  such  as  Modems  or  Multiplexers,  the  NCUs  can 
provide  the  automatic  switch-over  to  restore  a system  that  has  failed. 
Also,  the  same  capability  exists  for  alternate  routing  of  data  traffic  by 
the  NCUs  if  the  alternate  circuits  are  available. 

5.  IMPLEMENTATION  OF  THE  DNR  CONCEPT 

In  order  to  implement  the  DNR  concept  and  investigate  its  feasibility,  it 
is  necessary  to  fabricate  the  required  hardware  and  software,  or  a reason- 
able representation  of  it,  and  try  it  out  on  several  lines  of  a data 
communication  link.  As  far  as  possible,  this  implementation  should 
resemble  the  basic  product  in  its  function.  Implementation  will,  of 
course,  allow  for  experimentation,  and  modifications  would  be  expected  as 
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various  portions  of  the  concept  may  not  prove  feasible  in  an  operating 
environment. 

On  two  or  three  links  of  an  existing  communications  system,  MACIMS  for 
example,  such  a setup  would  require  at  most  two  NCUs  and  six  TIUs.  Instead 
of  a bidirectional  scanner  (BDS),  the  NCU  could  be  manually  switched 
between  the  three  or  so  lines  it  monitors  and  tests.  The  "NCU"  itself  need 
not  be  a single  unit  affair;  indeed,  it  could  be  an  array  of  modular  units 
(separate  printer,  keyboard,  microprocessor  and  memory,  test  signal  trans- 
mitter and  receiver,  real  time  deck,  etc).  Such  items  as  the  line 
selector,  scanner  mode  circuitry  and  self-test  circuitry  would  not  be 
mandatory  during  initial  implementation  tests.  This  "NCU"  should,  at  this 
time,  be  supplemented  with  test  equipment  such  as  a data  scope  and  3ER 
tester.  This  will  ensure  that  the  NCUs  properly  react  to  inputs  from  the 
lines,  modems,  multiplexers  and  other  communications  equipment  it  moni- 
tors. The  TIUs,  which  are  relatively  simple,  can  be  easily  fabricated, 
perhaps  with  some  of  the  automatic  functions  handled  manually.  ORACLE,  for 
this  implementation,  can  be  a portion  of  an  existing  Data  Computer  System 
and  can  be  accessed  by  a local  terminal.  The  overall  implementation  can  be 
carried  out  in  various  stages  (increasing  in  complexity)  as  the  concept 
proves  itself. 

6.  RECOMMENDATIONS. 

Downtime  of  a data  communication  system  can  be  reduced  by  a substantial 
amount  when  a first  level  (functional  level)  monitor  and  test  subsystem 
such  as  the  DNR  is  used  to  observe  trends  and  identify  potential  problems. 
These  trends  should  be  identified  early  enough  so  that  corrective  action 
can  be  taken  before  they  result  in  a fault  of  sufficient  magnitude  to  bring 
the  communication  system  down. 
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The  following  recommendations  are  made: 


a.  Comprehensive  acceptance  tests  for  compliance  with  specifications 
should  be  performed  on  all  lines  and  equipment  of  any  new  data  communica- 
tion systems  or  any  additional  lines  or  equipment  being  added  to  existing 
operating  systems.  All  lines  or  circuits  should  be  fully  characterized, 
through  testing,  before  becoming  part  of  the  communication  system. 
Admittedly,  this  exercise  has  always  been  considered  one  of  the  first 
things  done  (supposedly)  in  setting  up  or  adding  to  a system.  However, 
actual  practice  and  experience  indicate  that  this  is  not  always  the  case. 
It  is  reiterated  here  to  emphasize  its  importance. 

b.  All  future  exhibits,  STATEMENT  OF  REQUIREMENTS  (SOR),  STATEMENT  OF 
WORKS  (SOW),  etc.,  shall  provide  that  all  communication  equipments 
(Modems,  Muxs,  Line  Drivers,  etc.)  have  built  into  them  the  capability  for 
self- tests  whose  outputs  appear  at  a set  of  connector  pins. 

c.  Provisions  should  exist  for  physical  access  and  electrical 
connection  to  all  major  subsystem  equipment  and  circuits  in  the  data 
communication  system  for  monitoring  and  testing  purposes. 

d.  Standards  should  be  developed  for  both  quality  and  availability  of 
line  and  equipment  used  in  data  communication  systems. 

e.  Installation  and  performance  standards  should  be  developed  for 
network  monitoring  subsystems. 

f.  A set  of  procedure!  and  reporting  techniques  should  be  investi- 
gated and  defined  for  network  monitoring. 

g.  Additional  investigations  should  be  made  into  the  use  of  software 
techniques,  such  as  COLTS,  and  the  use  of  Hardware /Software  trade-offs  in 
the  monitoring  problem.  In  this  connection,  some  of  the  existing  tech- 
niques such  as  the  eye-pattern  can  be  integrated  more  into  the  monitoring 
system  through  the  use  of  hardware  and  testing  algorithms. 
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h.  Consideration  should  be  given  to  a plan  for  testing  the  concept 
presented  in  this  report.  This  plan  could  follow  the  steps  described  in 
paragraph  5 above . 
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APPENDIX  A 


DATA  NETWORK  QUESTIONNAIRE 


A copy  of  the  questionnaire  that  was  sent  to 
the  various  ADP  - Communication  Networks. 


A-1 


Locacloa  

Data  Network  Identifier 


1.  Evaluation.  Users  overall  evaluation  of  data  network  performance. 


Satisf ac  tory 
Unsatisfactory 


(Check  one) 


Optional  remarks:  (Please  identify  those  features  which  are  primarily 

responsible  for  the  rating  assigned  above.)  


I 


I 

t 


2.  Data  Network  Equipment  Configuration  Diagram.  Provide  a diagram 
depicting  interconnection  of  network  equipment.  Identify  each  equip- 
ment item  shown  by  manufacturer  and  model.  Identify  each  leased 
equipment  item  and/or  transmission  path.  When  multiple  vendor's 
equipment  are  present,  identify  those  items  provided  by  each.  A 
sample  diagram  is  attached  which  amplifies  and  displays  the  desired 
level  of  detail. 


3.  Operating  Conditions.  Identify  the  conditions  of  data  network 
operation,  e.g.,  hours  per  day,  and  days  per  week.  If  all  terminals 
do  not  operate  during  the  same  period  of  time,  identify  the  operating 
conditions  for  each  terminal.  Identify  each  bit  rates,  traffic  flow 
statistics,  response  times  and  communications  protocol  concept,  etc. 


A-2 


A.  Fault  Detection.  Does  any  of  the  data  network  equipment 
automatically  indicate  the  presence  of  errors,  equipment  mal- 
function, or  substandard  performance? 


(Check  one) 


If  yes,  please  identify  the  type  and  location  of  indication,  the 
equipment  that  provides  the  Indication,  and  type  of  personnel 
most  likely  to  receive  the  indication,  l.e.,  operators,  main- 
tenance personnel,  etc.  


5.  Data  Network  Performance  History.  Is  performance  data 
available  (outage  records  or  trouble  tickets)? 


Yes 

No 


(Check  one) 


If  yes,  provide  a synposls  of  equipment  and/or  transmission  path  malfunctions 
for  the  past  365  days.  


6.  Troubleshooting  Procedures.  Provide  a copy  of  any  instructions  provided 
to  assist  operators  or  maintenance  personnel  in  the  identification  of  mal- 
functioning path(s). 


7.  Diagonostic  Equipment.  Are  any  diagnostic  aids  provided  for  fault 
isolation  purposes  (hardware  or  software)? 


Yes 


No 


(Check  one) 


If  yes,  address  the  following: 


a.  Are  the  diagnostic  aids  included  in  the  data  processing  equipment? 


Yes 

No 


If  yes , cojnp the  following: 


(Check  one) 


(1)  Describe  the  features  provided  by  each  applicable  piece 


(2)  Briefly  discuss  the  utility  of  each  feature. 


b.  Are  the  diagnostic  aids  contained  in  equipment  separate  from  the  data 
processing  equipment? 


(Check  one) 

If  yes,  complete  the  following: 


(1)  Describe  the  features  provided  by  ea~h  applicable  piece  of 
equipment.  
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(2)  Briefly  discuss  the  utility  of  each  feature. 


8.  Diagnostic  Capability.  From  an  operator's  point  of  view,  identify  the 
diagnostic  capabilities  required  for  satisfactory  data  network  performance, 
e.g.,  additional  hardware/software  features  (specify),  fault  isolation, 
procedures,  etc.  Also,  Include  the  desired  method  for  implementation,  i.e., 
as  part  of  the  data  processing  equipment  or  separate  from  the  data  processing 
equipment.  


9.  Point  of  Contact.  Identify  an  individual  as  a point  of  contact  for  information 
regarding  the  data  network. 

NAME:  

AUTOVON  NUMBER: 


10.  Other  Comments.  Include  other  comments  deemed  appropriate 
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APPENDIX  B 


DISCUSSION  OF  ANALOG  CIRCUIT 
PARAMETERS  AND  DIGITAL  DATA 
TRANSMISSION  PERFORMANCE 


1.  GENERAL.  In  the  past  communication  systems  were  used  primarily  for 
exchange  of  voice  frequency  information,  and  the  characteristics  of  a 4 KHz 
voice  channel  were  not  critical.  Imperfections  in  the  communications 
system  added  distortion  to  the  voice  information,  but  usually  the  informa- 
tion could  be  understood.  The  human  ear  is  a very  forgiving  receiver. 
Terminal  equipment  sophistication  and  changes  in  our  defense  posture  have 
placed  additional  requirements  on  our  communication  systems.  More  and  more 
the  4 KHz  voice  channel  is  used  to  pass  digital  data  information,  and 
digital  data  signals  are  more  sensitive  to  perturbations  of  some  analog 
parameters  than  are  voice  signals. 


Every  year  the  amount  of  digital  data  information  passed  on  communication 
systems  has  steadily  increased.  This  has  been  accomplished  primarily 
through  an  increase  in  the  speed  at  which  the  data  information  is  passed. 
High  data  rates  have  been  found  to  be  extremely  sensitive  to  perturbations 
of  some  analog  parameters. 

This  Appendix  will  discuss  the  relationship  between  analog  circuit  para- 
meters and  digital  data  transmission  performance. 


As  a start,  the  parameters  in  DCAC  310-70-1,  Supplement  1,  were  studied  for 
correlation  to  digital  data  transmission  performance.  Then  the  area  of 
spectrum  ratios  was  investigated. 
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2.  PARAMETERS  NOT  NORMALLY  CONSIDERED  CRITICAL:  Longitudinal  balance, 
impedance,  frequency  translation,  and  single  tone  interference  parameters 
are  not  considered  critical  because: 
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a.  They  are  very  seldom  found  out  of  tolerance  or 

b.  Even  when  they  are  out  of  tolerance,  they  don't  appear  to 
significantly  affect  digital  data  transmission  performance. 

1)  Longitudinal  Balance. 

Longitudinal  balance  is  an  indication  of  the  difference  in 
impedance  between  each  conductor  and  ground  of  a balanced  line.  The 
greater  the  difference  in  impedance,  the  more  susceptible  the  line  is  to 
feedover  from  other  sources.  This  may  show  up  as  crosstalk,  increased  idle 
channel  noise  (ICN),  or  phase  jitter. 

No  past  studies  have  been  found  that  show  a direct  correla- 
tion between  longitudinal  balance  and  digital  data  transmission  perfor- 
mance. Also  past  Scope  Creek  evaluations  show  that  rarely  does  longitudi- 
nal balance  not  meet  the  MIL  Standard  of  40  dB.  For  these  reasons, 
longitudinal  balance  is  not  considered  a critical  test. 

2)  Impedance. 

The  impedance  test  measures  the  impedance  of  the  VF  channel 
input  and  output  at  the  terminal.  It  looks  at  the  frequency  band  300-3400 
Hz.  Mismatches  in  channel  Impedance  could  result  in  poor  frequency 
response,  signal  power  loss,  or  inaccurate  signal  level  measurements.  This 
test  by  itself  is  not  considered  critical.  It  could  best  be  used  as  a 
troubleshooting  device.  Past  Scope  Creek  reports  have  indicated  that  this 
is  normally  not  a problem. 
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3)  Frequency  Translation. 


Frequency  translation  is  a measure  of  the  change  in  fre- 
quency of  a signal  as  it  travels  over  a communications  channel.  In 
general,  modems  can  track  frequency  translation  and  few  errors  will  occur 
until  a certain  threshold  is  reached.  After  this  point,  synchronism  is 
lost  and  throughput  is  reduced  to  zero  percent.  Most  modems  have  a 
threshold  greater  than  10  Hz  and  very  rarely  will  the  frequency  translation 
be  greater  than  this. 

4)  Single  Tone  Interference 

This  test  is  aimed  at  detecting  low  level  single  tones  that 
impair  the  quality  of  voice  transmission.  Any  single  tone  with  a high 
enough  level  to  significantly  affect  digital  data  transmission  performance 
will  be  detected  when  making  ICN  measurements.  This  test  can  be  used  as  a 
troubleshooting  device  but  in  itself  is  not  considered  a critical  analog 
test . 

3.  PARAMETERS  THAT  MAY  BE  CRITICAL:  The  following  parameters  have 
demonstrated  that  they  can  significantly  affect  digital  data  transmission 
performance.  Normally  they  are  compensated  for  and  thus  are  only  a 
contributing  factor.  If  allowed  to  degrade,  these  parameters  will  become 
critical  and  cause  poor  digital  data  transmission  performance. 

a.  Envelope  Delay  Distortion. 

Envelope  delay  distortion  does  not  normally  result  in  poor 
digital  data  transmission  performance  by  itself.  What  it  does  is  make  the 
modem  more  susceptible  to  other  degrading  factors.  In  one  study  it  was 
found  that  a circuit  that  met  C2  specifications  for  delay  distortion  needed 
a signal  to  noise  (S/N)  of  21  dB  to  obtain  a bit  error  rate  (BER)  of  10"^. 


C2  specifications  are  shown  in  Table  1.  This  was  for  one  particular  modem 

at  9600  bps.  The  same  modem  on  a circuit  that  did  not  meet  C2  specif ica- 
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tions  needed  a S/N  of  25  dB  to  obtain  a BER  of  10  . The  same  relationship 

is  true  for  impulse  noise  and  delay  or  phase  jitter  and  delay. 

TABLE  1 


IC2  Envelope  Delay  Specifications 


1000  - 2600  Hz:  less  than  500  usee 


600  - 2500  Hz:  less  than  1500  usee 
500  - 2800  Hz:  less  than  3960  usee  j 

I 

I 

1 

Another  reason  that  this  test  is  not  considered  critical  is  that  ■ 

delay  distortion  is  fairly  easily  compensated  for.  External  delay  equal- 

i 

tzers  can  be  installed  on  most  circuits  and  most  newer  modems  have  internal 
equalizers  that  automatically  compensate  for  delay  distortion. 


b.  Frequency  Response. 

The  comments  made  about  envelope  delay  distortion  are  also 
applicable  to  frequency  response.  Frequency  response  in  itself  will  not 
normally  degrade  digital  data  transmission  performance  significantly.  It 
will  make  the  circuit  more  or  less  susceptible  to  changes  in  other  analog 
parameters  such  as  phase  jitter,  S/N,  or  impulse  noise.  Also,  external 
amplitude  equalizers  or  automatic  equalizers  in  the  modems  are  often  used 
to  correct  any  frequency  response  problems. 
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c.  Harmonic  and  Intermodulation  Distortion. 


Harmonic  and  intermodulation  distortion  are  random  error  causing 
mechanisms.  Studies  show  that  these  are  not  significant  factors  in  data 
performance  because  in  most  cases,  their  measured  value  is  at  least  30  dB 
below  the  fundamental  test  tone. 

d.  Net  Loss  and  Net  Loss  Variation. 

Net  loss  is  not  normally  a problem  single  pads  or  amplifiers  can 
be  used  to  compensate  for  it.  Net  loss  variation  may  be  a problem  due  to 
the  dynamic  range  of  the  modem.  If  the  data  signal  is  near  the  edge  of  the 
dynamic  range  of  the  modem,  the  variations  in  the  net  loss  could  cause  the 
data  signal  to  be  outside  that  dynamic  range,  resulting  in  errors.  Instal- 
lers or  users  tend  to  set  their  modems  so  that  the  data  level  is  on  the  high 
end  of  the  modem.  This  provides  the  user  with  the  greatest  fade  margin  but 
variations  of  the  data  level  on  the  plus  side  will  put  the  data  signal 
outside  the  dynamic  range  of  the  modem  and  cause  errors.  Users  or 
Installers  should  check  the  incoming  data  signal  level,  its  variations,  and 
the  dynamic  range  of  the  modem  and  make  the  necessary  adjustments. 

e.  Composite  Data  Transmission  Level. 

Generally  the  composite  data  transmission  level  is  set  at  -13 
dBmO.  This  may  be  critical  because  this  composite  level  is  one  of  the 
factors  that  determines  the  signal  to  noise  ratio  (S/N)  and  S/N  is  critical 
to  data  performance.  This  test  is  normally  used  for  level  discipline. 

4.  PARAMETERS  THAT  ARE  CRITICAL:  The  following  parameters  are  considered 
critical  because  if  degraded,  they  will  adversely  affect  data  transmission 
performance. 
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a.  Idle  Channel  Noise. 

Idle  channel  noise  (ICN)  is  directly  related  to  signal  to  noise 
ratio.  Assuming  the  data  signal  remains  at  a fairly  constant  level,  ICN 
can  be  used  as  a good  indicator  of  data  performance.  S/N  is  a composite 
indicator;  anything  that  affects  the  signal  level  or  the  noise  will  be 
measured.  Problems  with  longitudinal  balance,  single  tone  interference, 
harmonic  or  intermodulation  distortion  will  increase  the  ICN  or  decrease 
the  S/N  ratio.  Impedance,  net  loss  variation,  and  composite  data  level 
will  affect  the  signal  level  and  hence  the  S/N  ratio.  Specific  require- 
ments will  depend  on  the  data  transmission  rate  and  type  of  modem  used  but 
in  general  a S/N  of  26  dB  will  provide  a BER  of  10~^  or  better. 

b.  Phase  Jitter. 

Phase  jitter's  effect  on  digital  data  transmission  performance 
depends  largely  on  the  type  of  modulation  used  in  the  modem.  A modem  that 
uses  phase  modulation  will  be  affected  more  by  phase  jitter  than  a modem 
that  uses  amplitude  modulation.  One  study  by  RADC,  published  in  March 
1971,  looked  at  13  different  modems  using  different  modulation  schemes  and 
came  to  the  conclusion  that  if  the  peak-to-peak  phase  deviation  is  greater 
than  5 degrees,  a BER  exceeding  10“^  must  be  tolerated.  A study  of  data 
transmission  on  Autovon  circuits  csune  to  the  same  conclusion  but  indicated 
that  newer  modems  may  very  well  make  a BER  of  10  feasible.  A more  recent 
study  on  an  ICC  modem  96  Multi  Mode  indicated  that  this  modem  would  operate 
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with  a BER  of  10  with  peak-to-peak  phase  jitter  (60  Hz)  not  greater  than 
30  degrees.  As  the  frequency  of  the  phase  jitter  decreases  from  60  Hz,  BER 
increases  significantly.  Less  than  18  degrees  peak-to-peak  phase  jitter  at 
30  Hz  is  needed  in  order  to  obtain  a BER  of  10”^. 


c.  Spectrum  Ratio. 
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[ 

i 
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A spectrum  ratio  test  is  very  similar  to  a S/N  test.  A 4 KHz  VF 
channel  is  divided  into  an  arbitrary  number  of  equal  slots  and  sampled 
individually.  Forty  slots  of  100  Hz  each  is  a convenient  method.  The 
spectrum  ratio  is  obtained  by  taking  the  composite  amplitude  of  certain 
slots  and  dividing  this  by  the  composite  amplitude  of  certain  other  slots. 
Electronic  Systems  Division  (ESD)  evaluated  some  spectrum  ratios  in  an 
operational  environment.  They  used  the  10th,  15th,  and  20th  slot  for  the 
numerator  slots  and  the  33rd,  3^th,  and  35th  slots  for  the  denominator 
slots.  The  numerator  slots  were  in  the  middle  of  the  channel  where  the 
data  signal  should  be  and  the  denominator  slots  were  near  the  edge  of  the 
channel  where  there  should  only  be  noise.  The  spectrum  ratio  was  obtained 
by  dividing  the  composite  amplitude  of  the  numerator  slots  by  the  composite 
amplitude  of  the  denominator  slots.  The  slots  for  this  spectrum  ratio  were 
chosen  with  the  idea  of  testing  fully  loaded  VFCTs.  Different  types  of 
data  signals  would  require  using  different  slots.  The  ESD  evaluation 
concluded  that  the  spectrum  ratio  did  measure  a relative  S/N  ratio  and  we 
have  already  concluded  that  S/N  ratio  is  a good  indicator  of  data  transmis- 
sion performance.  More  study  is  needed  to  determine  the  optimum  selection 
of  slots  for  different  types  of  data  signals  and  the  threshold  settings. 
The  major  advantage  of  this  test  is  that  it  can  be  performed  in  service; 
there  is  no  interruption  of  service  to  the  customer. 

d.  Phase  Hits. 

Phase  hits  are  a burst  error  mechanism  which  can  substantially 
affect  the  BER,  especially  if  the  modem  uses  phase  modulation.  A threshold 
of  15  degrees  for  a hit  and  100  hits  in  15  minutes  are  usually  used  as 
standards.  Studies  indicate  that  phase  hits  are  a frequent  problem  on  data 

circuits.  One  study  correlating  phase  hits  and  BER  had  means  of  1.2  to  300 
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hits  resulting  in  10”  to  10“  BER  respectively  for  a data  rate  of  9600 
bps.  The  modem  used  in  this  study  used  a multi-level  vestigial  sideband 
type  of  modulation.  This  means  that  phase  hits  can  be  used  as  an  indicator 
of  data  performance  but  it  is  not  a composite  parameter.  Phase  hits  may 
not  always  be  a problem,  since  other  parameters  could  be  degrading  data 
performance . 


I 
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e.  Impulse  Noise. 


Impulse  noise  is  definitely  a cause  of  errors  in  digital  data 
transmission.  The  number  of  errors  caused  depends  basically  on  the 
amplitude,  rate,  and  duration  of  the  noise  pulse.  Impulse  noise  is  a 
predominant  factor  on  low  speed  data  circuits,  e.g.,  2^100  bps  or  less.  At 
higher  bit  rates,  other  analog  parameters  become  significant  also. 


Another  factor  to  consider  is  that  impulse  hits  appear  to  occur 


in  bursts  or  clusters.  This  means  that  even  though  the  BER  may  be  greater 
,-5 


than  10  , the  throughput  may  still  be  acceptable.  Thus,  impulse  noise  can 

be  considered  a good  indicator  of  data  performance. 


5.  CONCLUSION:  Longitudinal  balance,  impedance,  frequency  translation, 
and  single  tone  interference  are  parameters  that  are  not  critical  to 
digital  data  transmission  performance.  Past  experience  has  shown  that 
these  parameters  are  rarely  out  of  tolerance  or  that  digital  data  perfor- 
mance is  relatively  insensitive  to  them.  Only  if  they  are  extremely  bad 
will  they  affect  digital  data  performance. 


Envelope  delay  distortion,  frequency  response,  harmonic  distortion, 
intermodulation  distortion,  net  loss,  net  loss  variation,  and  composite 
data  transmission  level  may  be  critical  parameters  depending  on  their  value 
and  the  value  of  other  parameters.  Envelope  delay  and  frequency  response 
can  be  compensated  for  by  the  use  of  equalizers.  Harmonic  and  intermodula- 
tion distortion  can  be  a problem  if  there  is  too  much  nonlinearity  in  the 
circuit.  This  can  be  detected  when  determining  the  S/N  ratio  or  spectrum 
ratio.  Net  loss  is  not  normally  a problem  after  installation  but  net  loss 
variation  can  be.  This  will  also  show  up  in  the  composite  data  transmis- 
sion level  test.  The  variation  of  level  can  be  a problem  if  it  falls 
outside  the  dynamic  range  of  the  modem. 
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There  are  five  parameters  or  tests  that  are  critical  to  digital  data 
transmission  performance. 

Impulse  noise  is  basically  a burst  error  mechanism  so  that  while  the 
BER  may  be  degraded  significantly,  the  throughput  may  not  be  as  bad  as 
one  would  expect.  Impulse  noise  is  a dominate  factor  at  lower  data  rates 
2400  bps).  At  higher  data  rates  ("»  9600  bps)  it  becomes  an  equal  fac- 
tor along  with  S/N  ratio.  Between  2400  bps  and  9600  bps  it  gradually 
becomes  less  significant,  depending  on  other  parameter  values. 

Phase  hits  are  also  a burst  error  mechanism.  This  parameter  is  more 
dominate  on  systems  that  use  phase  modulation.  But  phase  hits,  like 
impulse  hits,  are  usually  random  in  nature.  This  would  be  useful  as  a 
troubleshooting  device  but  not  very  useful  as  a detection  or  monitoring 
device. 

Phase  jitter  is  a critical  factor,  again,  more  so  for  phase  modulated 
systems.  Not  only  is  the  peak-to-peak  phase  deviation  important,  so  is  the 
frequency  of  the  phase  jitter.  Studies  indicate  that  the  lower  the 
frequency  of  the  phase  jitter,  the  more  disruptive  to  data  transmission 
performance.  Newer  modems,  such  as  the  ICC  96  Multi  Mode,  are  less 
sensitive  to  phase  jitter  than  the  older  modems.  They  can  operate  at  9600 
bps  when  it  experiences  less  than  30  degrees  phase  jitter  (60  Hz). 

At  higher  data  rates,  idle  channel  noise  (ICN)  plays  an  increasing 
important  role.  ICN  is  an  important  component  of  the  S/N  ratio,  and  the 
S/N  ratio  is  a good  indicator  of  data  transmission  performance.  ICN  is  a 
composite  parameter;  anything  that  affects  the  idle  noise  of  a system  such 
as  phase  jitter  or  crosstalk  will  show  up  here.  Present  modems  require 
approximately  a 26  dB  S/N  ratio  for  operation. 
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With  the  exception  of  the  composite  data  transmission  level  test,  the 
spectrum  ratio  test  is  the  only  one  that  can  be  performed  in  service. 
Indications  are  that  the  spectrum  ratio  is  correlatable  to  S/N  ratio  which 
is  correlatable  to  BER  or  data  performance.  More  study  is  needed  on 
spectrum  ratios,  but  it  appears  to  have  excellent  possibilities  as  a 
detection  and  monitoring  device. 

After  spectrum  ratio  limits  have  been  determined  for  different  data 
signals,  it  can  be  monitored  on  a periodic  basis.  This  would  help  identify 
data  problems  before  an  actual  outage  occurs . 


In  conclusion,  the  analog  parameters  that  affect  digital  data  trans- 
mission the  most  are  spectrum  ratio,  idle  channel  noise  (S/N  ratio),  phase 
hits,  phase  jitter,  and  impulse  noise.  Important  analog  parameter  tests 
are:  spectrum  ratio,  idle  channel  noise,  phase  jitter,  phase  hits,  impulse 
noise,  envelope  delay  distortion,  composite  data  level,  and  frequency 
response.  These  should  be  used  in  conjunction  with  some  digital  parameter 
tests  such  as  BER  or  parity  error  checking.  The  spectrum  ratio  can  be  used 
in  service  on  the  analog  signal  to  give  an  indication  of  the  quality  of  the 
digital  data  transmission  performance.  The  other  analog  parameters  can  be 
used  for  troubleshooting  or  analysis. 

6.  RECOMMENDATIONS:  Studies  done  so  far  indicate  that  spectrum  ratios 
may  be  useful  for  monitoring  and  evaluating  digital  data  transmission 
performance.  Recommend  that  further  testing  be  done  in  this  area  to 
conclusively  define  the  correlation  between  spectrum  ratios  and  different 
types  of  digital  data  traffic. 
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